K8s 存储卷、ConfigMap 和 Secret

在 k8s 中目前个人认为最难的部分就是网络和存储了，网络是难搞懂的，就如之前的 ingress 一样，到目前都没有弄清楚，现在这里的存储也是只了解了一些简单的，如果想要真正的搞懂那需要不断的学习一些专业的存储知识。在 k8s 中还有 configmap 和 secret，用于我们修改应用程序的配置文件，这两者都是常用到的。

pv, pvc

一般情况下由专门的存储工程师创建出存储空间，交给 K8s 管理员创建出 Pv，K8s 使用者创建 Pvc，应用 Pv。Pvc 和 Pv 是一一对应关系，但是一个 Pvc 可以支持被多个 Pod 所使用。Pvc 如果成功绑定 Pv，那么状态就是 Bound，否则处于 Peding 状态。

创建 pv 示例：https://www.cnblogs.com/along21/p/10342788.html

emptyDir

无持久性，随着 Pod 的删除也会被删除。

创建 emptyDir

定义自助式 Pod 来测试 emptyDir。volumes.emptyDir.medium 可以指定存储的介质。sizeLimit 可以指定存储总量

kubectl explain pods.spec.volumes.emptyDir

KIND:     Pod
VERSION:  v1

RESOURCE: emptyDir <Object>

DESCRIPTION:
     EmptyDir represents a temporary directory that shares a pod's lifetime.
     More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir

     Represents an empty directory for a pod. Empty directory volumes support
     ownership management and SELinux relabeling.

FIELDS:
   medium	<string>
     What type of storage medium should back this directory. The default is ""
     which means to use the node's default medium. Must be an empty string
     (default) or Memory. More info:
     https://kubernetes.io/docs/concepts/storage/volumes#emptydir

   sizeLimit	<string>
     Total amount of local storage required for this EmptyDir volume. The size
     limit is also applicable for memory medium. The maximum usage on memory
     medium EmptyDir would be the minimum value between the SizeLimit specified
     here and the sum of memory limits of all containers in a pod. The default
     is nil which means that the limit is undefined. More info:
     http://kubernetes.io/docs/user-guide/volumes#emptydir

emptyDir.yaml 内容如下：volumes 中的内容会创建一个 emptyDir，可以限制 emptyDir 使用的大小，{} 代表不限制

apiVersion: v1
kind: Pod
metadata:
  name: runsha.yan.test.pod
  namespace: default
  labels:
    app: myapp
  annotations:
    create-by: runsha.yan
spec:
  containers:
  - name: myapp
    image: ikubernetes/myapp:v1
    imagePullPolicy: IfNotPresent
    ports:
    - name: http
      containerPort: 80
    volumeMounts:
    - name: html
      mountPath: /usr/share/nginx/html/
  - name: busybox
    image: busybox:latest
    imagePullPolicy: IfNotPresent
    volumeMounts:
    - name: html
      mountPath: /data/
    command: ["bin/sh"]
    args: ["-c", "while true; do echo $(date) >> /data/index.html; sleep 2; done"]
  volumes:
  - name: html
    emptyDir: {}

创建 Pod

kubectl apply -f emptyDir.yaml

查看内容

kubectl get pods
NAME                                      READY   STATUS    RESTARTS   AGE
runsha.yan.test.pod                       2/2     Running   0          7m

创建 NodePort Service 进行访问测试

nodePort.yaml

apiVersion: v1
kind: Service
metadata:
  name: runsha-yan-test-service
  namespace: default
spec:
  selector:
    app: myapp
  type: NodePort
  ports:
  - port: 80
    targetPort: 80
    nodePort: 30080

创建 Service

kubectl apply -f nodePort.yaml

查看状态

 rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl get svc
NAME                      TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                          AGE
runsha-yan-test-service   NodePort    172.16.3.101    <none>        80:30080/TCP                     7m

浏览器访问测试

可以看到每隔两秒内容进行增加。

进入容器查询内容

kubectl exec -it runsha.yan.test.pod -c  myapp -- /bin/sh

/ # cd /usr/share/nginx/html/
/usr/share/nginx/html # ls -la
total 20
drwxrwxrwx    2 root     root          4096 Jun 15 09:42 .
drwxr-xr-x    1 root     root          4096 Feb 25  2018 ..
-rw-r--r--    1 root     root         10121 Jun 15 09:54 index.html
/usr/share/nginx/html # echo "test" >>  test.html

浏览器访问 test 页面同样也可以看到内容

进入另一个容器

kubectl exec -it runsha.yan.test.pod -c busybox  -- /bin/sh

因为 busybox 容器和 myapp 共同挂载的一个 emptyDir，所以也可以看到之前在 myapp 容器内创建的 test.html 文件

/ # cd /data/
/data # ls -la
total 24
drwxrwxrwx    2 root     root          4096 Jun 15 09:55 .
drwxr-xr-x    1 root     root          4096 Jun 15 09:42 ..
-rw-r--r--    1 root     root         12064 Jun 15 09:56 index.html
-rw-r--r--    1 root     root             5 Jun 15 09:55 test.html

hostPath

创建 hostPath

与docker 中的映射一致，挂载一个宿主机的目录或文件或者其他格式。具有一定的持久性，但是如果节点出现问题，那么持久性将受到影响。hostPath type类型支持好几种方式，例如：DirectoryOrCreate（目录，没有则创建），Directory（目录），FileOrCreate（文件，没有则创建），File（文件），Socket，CharDevice，BlockDevice。

apiVersion: v1
kind: Pod
metadata:
  name: runsha-yan-test-pod-hostpath
  namespace: default
spec:
  containers:
  - name: myapp
    image: ikubernetes/myapp:v1
    volumeMounts:
    - name: html
      mountPath: /usr/share/nginx/html/
  volumes:
  - name: html
    hostPath:
      path: /tmp/
      type: DirectoryOrCreate

这是因为没有宿主机 ssh 的权限，所以就不验证效果了。

ConfigMap

ConfigMap 存储信息为明文, 定义一个配置文件

server {
      server_name test.runsha.yan.com;
      listen 80;
      root /data/web/html;
}

创建 configmap

ubectl create configmap runsha-yan-test-configmap --from-file=./www.config

查看 configmap

1. 查看列表

 ✘ rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl get cm
NAME                        DATA   AGE
runsha-yan-test-configmap   1      1m

2. 查看 yaml 格式

   ✘ rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl get cm runsha-yan-test-configmap -o yaml
   apiVersion: v1
   data:
     www.config: |
       server {
             server_name test.runsha.yan.com;
             listen 80;
             root /data/web/html;
       }
   kind: ConfigMap
   metadata:
     creationTimestamp: "2019-06-16T03:29:07Z"
     name: runsha-yan-test-configmap
     namespace: default
     resourceVersion: "52482318"
     selfLink: /api/v1/namespaces/default/configmaps/runsha-yan-test-configmap
     uid: e5f0531e-8fe6-11e9-bbd9-00163e0aa20d

3. 查看详细信息

   kubectl describe cm runsha-yan-test-configmap

注入到容器

1）环境变量

使用环境变量的方式注入，前提是应用程序能解析并应用环境变量（如果程序不支持读取环境变量的配置，那么即使注入也没用）。使用命令行创建一个 名称为 nginx-config 的 configmap

kubectl create configmap nginx-config --from-literal=nginx_port=80 --from-literal=server_name=runsha.yan.test.configmap

 ✘ rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl  get cm
NAME                        DATA   AGE
nginx-config                2      14s
runsha-yan-test-configmap   1      35m

查看详情

 rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl describe cm nginx-config configMapEnv.yaml
Name:         nginx-config
Namespace:    default
Labels:       <none>
Annotations:  <none>

Data
====
nginx_port:
----
80
server_name:
----
runsha.yan.test.configmap
Events:  <none>

创建 Pod 使用 env 方式注入, configMapKeyRef 中，name 为 configmap 的名称，key 为配置的名称。这样就会在容器中存在 NGINX_SERVER_PORT 和 NGINX_SERVER_NAME 两个环境变量，并且值为我们刚刚在 configmap 中创建的。

apiVersion: v1
kind: Pod
metadata:
  name: runsha.yan.test.configmap.pod
  namespace: default
  labels:
    app: myapp
  annotations:
    create-by: runsha.yan
spec:
  containers:
  - name: myapp
    image: ikubernetes/myapp:v1
    imagePullPolicy: IfNotPresent
    ports:
    - name: http
      containerPort: 80
    env:
    - name: NGINX_SERVER_PORT
      valueFrom:
        configMapKeyRef:
          name: nginx-config
          key: nginx_port
    - name: NGINX_SERVER_NAME
      valueFrom:
        configMapKeyRef:
          name: nginx-config
          key: server_name

 rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl apply -f configMapEnv.yaml
pod/runsha.yan.test.configmap.pod created

 ✘ rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl get pods
NAME                                      READY   STATUS    RESTARTS   AGE
runsha.yan.test.configmap.pod             1/1     Running   0          13s

进入容器查看环境变量是否是我们刚才设置的

 ✘ rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl exec -it runsha.yan.test.configmap.pod -- /bin/sh
/ # echo $NGINX_SERVER_PORT
80
/ # echo $NGINX_SERVER_NAME
runsha.yan.test.configmap

查看我们的环境变量，的确已经看到了期待的结果。接下来使用 edit 来编辑 configmap，然后再来观察现象。

rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl edit cm nginx-config

将 server_name 的值修改为 server_name: runsha.yan.test.configmap.update, 重新进入容器查看环境变量信息。会发现结果没有发生改变。

 rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl exec -it runsha.yan.test.configmap.pod -- /bin/sh
/ # echo $NGINX_SERVER_NAME
runsha.yan.test.configmap

使用 edit 修改 configmap，再次进入 Pod 发现环境变量的值没有发生变化，原因是使用环境变量的方式注入，只会在容器启动的时候才会注入生效，之后修改 configmap 是没有效果的。

2）使用存储卷

需要创建一个存储卷，类型为 configmap，再将创建的 configmap 关联至容器中。进入 Pod 会看见文件名字为键名，键值为文件内容的文件已经被注入到指定挂载的目录了。configmap 就使用上面创建的 nginx-config

apiVersion: v1
kind: Pod
metadata:
  name: runsha.yan.test.configmap.mount.pod
  namespace: default
  labels:
    app: myapp
  annotations:
    create-by: runsha.yan
spec:
  containers:
  - name: myapp
    image: ikubernetes/myapp:v1
    imagePullPolicy: IfNotPresent
    ports:
    - name: http
      containerPort: 80
    volumeMounts:
    - name: nginxconfig
      mountPath: /etc/nginx/config.d/
      readOnly: true
  volumes:
  - name: nginxconfig
    configMap:
      name: nginx-config

 rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl apply -f configMapMount.yaml
pod/runsha.yan.test.configmap.mount.pod created

进入容器进行验证

 rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl exec -it runsha.yan.test.configmap.mount.pod -- /bin/sh
/ # cd /etc/nginx/config.d/
/etc/nginx/config.d # ls -la
total 12
drwxrwxrwx    3 root     root          4096 Jun 16 08:05 .
drwxr-xr-x    1 root     root          4096 Jun 16 08:05 ..
drwxr-xr-x    2 root     root          4096 Jun 16 08:05 ..2019_06_16_08_05_23.315302408
lrwxrwxrwx    1 root     root            31 Jun 16 08:05 ..data -> ..2019_06_16_08_05_23.315302408
lrwxrwxrwx    1 root     root            17 Jun 16 08:05 nginx_port -> ..data/nginx_port
lrwxrwxrwx    1 root     root            18 Jun 16 08:05 server_name -> ..data/server_name
/etc/nginx/config.d # cat server_name
runsha.yan.test.configmap.update/etc/nginx/config.d #

cat server_name 就会看到内容为 runsha.yan.test.configmap.update。文件的名字是 configmap 的键名，nginx_port 和server_name。文件的内容分别是 configmap 中对应键名的值。

使用 edit 修改 configmap，将erver_name 的内容改为 runsha.yan.test.configmap.mount ,再次进入Pod 发现对应修改的 key 的文件的内容已经发生了变化。

Secret

创建 Secret

secret 和 configmap 相比，secret 不是以明文方式保存信息的，而是以 base64 的方式。secret 共有三种类型，一种是 docker-register，用于 kubelet 在私有仓库拉取镜像时使用。一种为 tls，用于保存证书信息。其余的都可以保存为 generic。

创建 generic 类型的 secret

rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl create secret generic runsha.yan.test.mysql.pass --from-literal=password=123456

上述命令中我们创建了一个名字叫做 runsha.yan.test.mysql.pass generic 类型的 secret，键名 为 password 值为 123456。查看详细信息：

rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl get secrets
NAME                                    TYPE                                  DATA   AGE
runsha.yan.test.mysql.pass              Opaque                                1      2m

查看 yaml 格式的内容，发现看见的密码是 base64 转码之后的。

 rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl get secrets runsha.yan.test.mysql.pass -o yaml
apiVersion: v1
data:
  password: MTIzNDU2
kind: Secret
metadata:
  creationTimestamp: "2019-06-16T08:16:29Z"
  name: runsha.yan.test.mysql.pass
  namespace: default
  resourceVersion: "52525316"
  selfLink: /api/v1/namespaces/default/secrets/runsha.yan.test.mysql.pass
  uid: 0b45f531-900f-11e9-a6ce-00163e00c9c1
type: Opaque

注入容器

1）环境变量

使用我们刚刚创建的 runsha.yan.test.mysql.pass 的 secret

 rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl describe secrets runsha.yan.test.mysql.pass
Name:         runsha.yan.test.mysql.pass
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
password:  6 bytes

创建 Pod

apiVersion: v1
kind: Pod
metadata:
  name: runsha.yan.test.secret.env.pod
  namespace: default
  labels:
    app: myapp
  annotations:
    create-by: runsha.yan
spec:
  containers:
  - name: myapp
    image: ikubernetes/myapp:v1
    imagePullPolicy: IfNotPresent
    ports:
    - name: http
      containerPort: 80
    env:
    - name: MYSQL_PASSWORD
      valueFrom:
        secretKeyRef:
          name: runsha.yan.test.mysql.pass
          key: password

rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl apply -f secretEnv.yaml

进入容器后查看注入的信息是明文的

 ✘ rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl exec -it runsha.yan.test.secret.env.pod -- /bin/sh
/ # echo $MYSQL_PASSWORD
123456

2）使用存储卷

和 configmap 一样，也是将存储卷转换为文件的方式注入

apiVersion: v1
kind: Pod
metadata:
  name: runsha.yan.test.secret.mount.pod
  namespace: default
  labels:
    app: myapp
  annotations:
    create-by: runsha.yan
spec:
  containers:
  - name: myapp
    image: ikubernetes/myapp:v1
    imagePullPolicy: IfNotPresent
    ports:
    - name: http
      containerPort: 80
    volumeMounts:
    - name: mysql-pass
      mountPath: /etc/mysql/
      readOnly: true
  volumes:
  - name: mysql-pass
    secret:
      secretName: runsha.yan.test.mysql.pass
      items:
      - key: password
        path: config/mymysql

items 中的 key 是需要映射的 configmap 中的键名，path 是键值的位置。进入容器进行查看，进入到 /etc/mysql/config/ 目录下，就可以看到 一个名字为 mymysql 的文件，内容是键值 123456。

 rex@yanrunshadeMacBook-Pro  ~/Documents/Projects/k8s  kubectl exec -it runsha.yan.test.secret.mount.pod -- /bin/sh 
/ # cd /etc/mysql/config/
/etc/mysql/..2019_06_16_08_56_59.776690184/config # ls -la
total 4
drwxr-xr-x    2 root     root            60 Jun 16 08:56 .
drwxr-xr-x    3 root     root            60 Jun 16 08:56 ..
-rw-r--r--    1 root     root             6 Jun 16 08:56 mymysql
/etc/mysql/..2019_06_16_08_56_59.776690184/config # cat mymysql
123456/etc/mysql/..2019_06_16_08_56_59.776690184/config #